1. Technical Field
The present invention relates generally to a system and method for the functional testing of distributed, software components during unit and integration testing and more particularly to a system and method of testing components, which combines test generation and test execution technology with commercial UML modeling tools to form a design-based testing environment.
2. Description of the Related Art
Increasing numbers of software developers are using the Unified Modeling Language (UML) and associated visual modeling tools as a basis for the design and implementation of their distributed, component-based applications. At the same time, it is necessary to test these components, especially during unit and integration testing.
While standardized testing strategies and tools have been available for IC (hardware) components for many years, the research and development of standardized testing techniques and tools for distributed software components has just begun. Three key technological factors are not only allowing developers to design and implement components, but at the same time, they are contributing towards the development of such testing strategies.
These factors include:
The definition of the Unified Modeling Language (UML), a standardized way of modeling the static structure and dynamic behavior of components and their interfaces;
The standardization of object-oriented middleware, for example, Microsoft""s Component Object Model/Distributed Component Object Model (COM/DCOM) and the Object Management Group""s (OMG) Common Request Broker Architecture (CORBA);
The continued refinement of standard object-oriented programming languages, such as Java and C++, and integrated development environments that provide extensive support for creating distributed components.
As a result, developers are implementing large numbers of components ranging from relatively simple graphical user interface (GUI) components to sophisticated server-side application logic.
After designing and coding each software component, developers perform unit testing to ensure that each component correctly implements its design and is ready to be integrated into a system of components. This type of testing is performed in isolation from other components and relies heavily on the design and implementation of test drivers and test stubs. The purpose of these test drivers is to stimulate the component under test via its interfaces, while test stubs receive the responses from the component under test via these interfaces when stimulated. New test drivers and stubs have to be developed to validate each of the components in the system.
After unit testing is concluded, the individual components are collated, integrated into the system, and validated again using yet another set of test drivers/stubs. At each level of testing, a new set of custom test drivers/stubs is required to stimulate the components. While each component may have behaved correctly during unit testing, it may not do so when interacting with other components. Therefore, the objective of integration testing is to ensure that all components interact and interface correctly with each other, that is, have no interface mismatches.
Basically, as developers are delivering these complex, server-side components, they must also ensure that each component is delivered with a concise and unambiguous definition of its interfaces, and the legal order in which operations may be invoked on them. Component interfaces and their protocol specifications are being described or modeled in a variety of ways. For example, in the case of the Enterprise Java Beans Specification, this is achieved through contracts and UML Sequence Diagrams (also known as Message Sequence Charts). While a Sequence Diagram is useful at describing a specific interaction scenario, it may require a large number of such diagrams to completely specify the interaction of a complex component with its client(s).
Over the years, there have been numerous papers dedicated to the subject of test data generation. Moreover, a number of tools have been developed for use within academia and the commercial market. These approaches and tools have been based on different functional testing concepts and different input languages, both graphical and textual in nature.
However, few received any widespread acceptance from the software development community at large. There are a number of reasons for this. First, many of these methods and tools require a steep learning curve and a mathematical background. Second, the modeling of larger systems beyond single components could not be supported, both theoretically and practically. Third, the design notation, which would be used as a basis for the test design, was often used only in a particular application domain, for example, SDL (Specification and Description Language) is used predominantly in the telecommunications and embedded systems domain.
However, with the widespread acceptance and use of UML throughout the software development community as well as the availability of suitable tools, this situation may be about to change. Offutt J., Abdurazik A., Generating Test Cases from UML Specifications, Proceedings of 2nd International Conference on UML ""99, October 1999, which presents an approach that involves generating test cases from UML Statecharts. However, their approach has a different focus in that they examine different coverage requirements. Their method also has several disadvantages. For example, it is only able to generate tests for a single component. Furthermore, it does not automate the test execution step in order for developers to automatically generate and execute their tests. In addition, it does not specifically address the problems and issues associated with modeling distributed, component-based systems.
Accordingly, there is a need for a system and method for the functional testing of distributed software components, which minimizes the testing costs, time and effort associated with initially developing customized test drivers and test stubs as well as repeatedly adapting and rerunning them for regression testing purposes at each level of integration.
The present invention is directed towards a system and method of defining the dynamic behavior of software components using Unified Modeling Language (UML)-based state machine representations, annotating them with testing specific information such as coverage criteria and data variations, interconnecting them in the case of multiple components, automatically deriving test cases from these state machine representations, and executing the test cases using a test execution environment developed specifically for testing components implemented using, for example, COM/CDOM and CORBA middleware.
In one aspect of the present invention, a method for testing distributed component-based software comprises the steps of: modeling dynamic behavior of a plurality of components with a plurality of UML-based state machine representations; creating a plurality of normalized state machine representations by converting the UML-based state machine representations into an internal representation; creating a global behavioral model by combining the normalized UML-based state machine representations; mapping the global behavioral model to a Test Specification Language (TSL) test design; processing the TSL test design using a Test Design Environment to produce a plurality of Interface Test Language (ITL) test cases; mapping these ITL test cases to standard object-oriented code using an ITL compiler and merging this code with additional code that is generated from an IDL (Interface Definition Language) file representing a plurality of sink objects; generating a plurality of executable test cases to form a test driver using a standard object-oriented language compiler by compiling the generated code with the test harness library code; and executing and monitoring these test cases with the test control center.